Database Handicapping Software- JCapper

JCapper Message Board

          JCapper 101
                      -- dumb question about using command line switch to automatically process old data files

Home Register
Log In
By dumb question about using command line switch to automatically process old data files
theiman
8/10/2018
1:25:47 PM		Hi Jeff,

In a previous message I mentioned a catastrophic hard drive failure where I lost all of my jcapper data. Thanks to wonderful people at HDW (and you) I was able to get the raw files for 3 years of data. As I slog through trying to recreate a datafile that uses all of the data by hand i.e. calculating all of the cards and getting that data into one file, I have been thinking that there has to be a better way.. As I program in R and in python I wondered if there is a way I could use the command line switches in a programmatic way to build my data file? Any thoughts on this or is the manual approach my only recourse? Thank you!!

Sincerely,

tom

Reply
jeff
8/11/2018
10:48:01 AM		--A horseplayer: "Hey Siri, build JCapper databases spanning three years from the files that guy Ron at HDW sent me."

--Siri: "Ok. I found the files from HDW. Building databases now."


Sadly, back in 2008-2009 when I was building out the HDW File Mgr, a routine designed to automate recovery of replacement data obtained from HDW never occurred to me.

The one thing along those lines that did occur to me -- and it didn't occur to me until several years later -- was the Switches Menu in the HDW File Mgr.

But the Switches on the menu are designed to automate the file download and file build process for current data -- not back data.

Trying to use the Switches Menu to process replacement data sent to you by HDW won't work because all of the Switches are programmed to download the rolling 45 days of data that HDW keeps on their file server.

Your replacement data is more than 45 days old and isn't on the HDW file server.


Back data is packaged differently... one big .zip file that contains all of the race files for a single month... and one big .zip file containing all of the chart files for a single month.



I hate to say it.

There really isn't a way around it.

Because your replacement data is more than 45 days old and isn't on the HDW file server:

You're going to have to run the whole thing manually.



These are the basic steps:

  1. Create the folder structure if you don't already have it. A yearly folder for each year. Four quarterly sub folders one level beneath each yearly folder.


  2. Working one quarter at a time:


    • Download the big .zip files to your hard drive using the links that Ron gave you.


    • Open up each big .zip file - identify the month and year for the files inside - then and extract the little .zip files inside out onto the appropriate quarterly folder.


    • Using the HDW File Mgr: Run an .XRD file build routine for the current quarter you are working on.


    • Using the HDW File Mgr: Run a .JCP file build routine for the current quarter you are working on.


    • Using the Main Module: Run a Build Database routine for the current quarter you are working on.

      --Hint: Run the very first db build routine in Mode 3. Then run each subsequent build routine in Mode 4.

      --Important: Do not allow your c:\JCapper\Exe\JCapper2.mdb file to exceed 2.0 gigabytes in size.

      Consider using one c:\JCapper\Exe\JCapper2.mdb file per calendar year. (That way your c:\JCapper\Exe\JCapper2.mdb file should never exceed 2.0 gigabytes in size.)

      --Hint: Each time you're ready to start the next year:

      A. Rename your c:\JCapper\Exe\JCapper2.mdb to archive it (changing the name to c:\JCapper\Exe\JCapper2_2017.mdb would be how I'd do it if the file contained starters from 2017.)

      B. After the rename in the above step, copy the JCapper2.mdb from your c:\JCapperBuild folder to your c:\JCapper\Exe\ folder.

      C. If you have a custom sql factor setup and/or html report layout: Don't forget to use the JCapper2 Import Module to import the sql factor setup and sql html report layout out of your old JCapper2.mdb file into your new JCapper2.mdb file.

      At this point you're ready to build the quarterly folders for the next year.



  3. Repeat as needed until there are no more quarterly folders left to build.


That's It!


-jp

.



~Edited by: jeff  on:  8/11/2018  at:  10:48:01 AM~

Reply
theiman
8/11/2018
11:05:04 AM		Hi Jeff,

Thank you!!

Sincerely,

tom


Reply
Reply
Copyright © 2018 JCapper Software              back to the JCapper Message Board              www.JCapper.com